Hypertext Classification Diploma thesis Hervé

Web Directories have been historically collected and updated by hand but this method is unsatisfactory for three reasons: A team of Web Surfers maintaining such a database should face the gigantism of the World network. Its size would thus be incompatible with the economic constraints of startups. Even the biggest team would not be able to trace all the changes on the Web and to keep the database up to date. Furthermore, categorization is a highly subjective task. Manual categorization is not a synonym for good categorization. However, automating the categorization of documents is a difficult task in the Web environment. The diversity of languages, topics and authorships prevents the traditional classification algorithms to work optimally. Fortunately, the internal HTML structure of the Web pages and the hyperlink graph structure of the Web are new sources of information that can be explored to improve automated Web page classification. In this diploma thesis, we carry on the work of Fürnkranz [8] about hypertext categorization. We investigate different classification techniques for categorizing hypertext documents. We target information rich text areas of the page and of its neighbors and we compare different methods for having those various features optimally help together for improved classification. We evaluate the heavy points and the weaknesses of the Hyperlink Ensembles and Meta Predecessor approaches. We explain how to choose a binarization algorithm between Round Robin and One Against All according to the behavior awaited. We compare two solutions for bringing features mined on different locations together, namely Tagging and Merging and we finally propose a model of hypertext classifier which combines the best characteristics of the methods we study. Our main result is a model of hyperlink based classifier that outperforms a text only classifier by almost 25% for the WebKB dataset.